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Two  Simple  Residual  Bounds 
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Let  A  be  a  Hermitian  matrix  with  eigenvalues  Ai  >  •  •  •  >  A„.  If  X  is  a  matrix 
with  orthonormal  columns  that  spans  an  invariant  subspace  of  A  and 


M  =  X^AX, 


(1) 


then  AX  -  XM  =  0. 

Now  suppose  that  the  columns  of  X  span  an  approximate  invariant  subspace 
of  A.  Then  the  matrix 

R  =  AX  -  XM 

will  be  small,  say  in  the  spectral  norm  ||  •  ||  dehned  by  ||i?||  =  max||2;||=i  ||i?x||, 
where  ||a;||  is  the  Euclidean  norm  of  xd  If  the  eigenvalues  of  M  are  >  •  •  •  >  //fc, 
then  we  should  expect  the  pi  to  be  near  k  of  the  Aj-.  The  problem  treated  in  this 
note  is  to  derive  a  bound  in  terms  of  the  matrix  R. 

An  important  result,  due  to  Kahan  [3]  (see  also  [6,  p.219])  states  that  there 
are  eigenvalues  Aj^ , .  .  .  ,  of  A  such  that 

1/^*  -  AjJ  <  ||i?||,  i  =  l,...,k.  (2) 

If  nothing  further  is  known  about  the  spectrum  of  A,  this  bound  is  generally 
satisfactory,  although  it  can  be  improved  somewhat  [5].  However,  it  frequently 
happens  (e.g.,  in  the  Lanczos  algorithm  or  simultaneous  iteration  [6,  ChA3-14]) 
that  we  know  that  n  —  k  oi  the  eigenvalues  of  A  are  well  separated  from  the 
eigenvalues  of  M\  specihcally,  if  we  know  that 

there  is  a  number  A  >  0  such  that  exactly 
n  —  k  of  the  eigenvalues  of  A  lie  outside  the  (3) 

interval  [pk  —  A,  +  A], 

then  the  bound  in  (2)  can  be  replaced  by  a  bound  of  order  ||i?|p.  Bounds  of  the 
kind  have  been  given  by  Temple,  Kato,  and  Lehman  (see  [6,  ChAO]  and  [1,  §6.5]). 

^In  fact,  the  choice  (1)  of  M  minimizes  ||R||,  although  we  will  not  make  use  of  this  fact  here. 
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Residual  Bounds 


Early  bounds  of  this  kind,  dealt  only  with  a  single  eigenvalue  and  eigenvector. 
Lehman’s  bounds  are  in  some  sense  optimal,  but  are  quite  complicated. 

The  purpose  of  this  note  is  to  give  two  other  bounds  derived  from  bounds  on 
the  accuracy  of  the  column  space  of  X  as  an  invariant  subspace  of  A.  They  are 
very  simple  to  state  and  yet  are  asymptotically  sharp.  In  addition  they  can  be 
established  by  appealing  to  results  readily  available  in  the  literature. 


Theorem  1.  With  the  above  dehnitions,  assume  that  A  and  M  satisfy  (3).  If 

_  \\m 


p  = 


8 


<  1, 


then  there  is  an  index  j  such  that  Aj, .  .  .  ,  £  {pk  ~  h,  +  8)  and 

1  —  8 

Proof.  Let  {X  Y)  be  unitary.  Then 

)  ,4,v  n  .  ^ 


S  N 


where  US'!!  =  ||i?||.  By  the  “sin  0”  theorem  of  Davis  and  Kahan  [2]  there  is  a 
matrix  P  satisfying 

||P(/  +  P^P)^||  <  p.  (4) 

such  that  the  columns  of 


X  =  [X  +  YP){I  +  P^P)-2 


(which  are  are  orthonormal)  span  an  invariant  subspace  of  A.  From  (4)  it  follows 
that 

ll^ll 

<  P, 


and  since  p  <  f 


1+  P 


\P\\  < 


(5) 


vT^' 

Let  Y  =  [Y  —  XP^)[I  +  PP^)~^ .  Then  {X  Y)  is  unitary.  Since  the  columns 
of  X  span  an  invariant  subspace  of  A,  we  have  Y^AX  =  0.  Hence 


)  ■4(-Y  i') 


M  0 
0  N 


Residual  Bounds 
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In  [7]  it  is  shown  that 

M  =  (/  +  P^P)h{M  +  S^P){I  +  P^P)-P 

The  eigenvalues  of  M  are  eigenvalues  of  A.  Since  p  <  1  it  follows  from  (2), 
they  lie  in  the  interval  [pk  —  h,  +  h),  and  hence  are  Aj, .  .  .  ,  for  some 

index  j.  By  a  result  of  Kahan  [4]  on  non-Hermitian  perturbations  of  Hermitian 
matrices, 

l/r.  -  <  11(7  +  P^P)^||||(/  +  P^P)-^||||P||||P||,  z  =  1, . . . ,  L 

The  theorem  now  follows  on  noting  that  ||(/  +  P^P)“2  ||  <  1  and  inserting  the 
bound  (5)  for  ||P||.  ■ 

Two  remarks.  First,  the  theorem  extends  to  operators  in  Hilbert  space,  pro¬ 
vided  X  (now  itself  an  operator)  has  a  hnite  dimensional  domain.  Second,  the 
bound  is  asymptotically  sharp,  as  may  be  seen  by  letting  X  =  {1  0)^  and 


(the  eigenvalues  of  A  are  asymptotic  to  and  1  —  e^). 

The  requirement  (3)  unfortunately  does  not  allow  the  eigenvalues  of  M  to  be 
scattered  through  the  spectrum  of  A.  If  we  pass  to  the  Frobenius  norm  dehned 
by  ||77||p  =  trace(X^X),  then  we  can  obtain  a  Hoffman- Wielandt  type  residual 
bound.  Specihcally,  if 

8  =  min{|Ai  —  Pj  \  :  \i  G  \{A),pj  G  A(Af)}  >  0,  (6) 

then  a  variant  of  the  sin  0  theorem  shows  that  there  is  a  matrix  P  satisfying 

||P(/  +  P^P)^||  <  ||P(/  +  P^P)^||f  < 

0 

such  that  the  columns  of 

1  =  (x  +  yp)(/  +  p^p)-^ 

span  an  invariant  subspace  of  A.  By  a  variant  of  Kahan’s  theorem  due  to  Sun 
[9,  8],  the  eigenvalues  Aj^ , .  .  .  ,  of  M  may  be  ordered  so  that 

,  'tu'.  -  h?  <  ll(z  +  r"r)*llll(z  +  r«P)-i||||Si|H|c||. 

\|  *=1 

Hence  we  have  the  following  theorem. 
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Residual  Bounds 


Theorem  2.  With  the  above  dehnitions,  assume  that  A  and  M  satisfy  (6).  If 


Pf 


PIIf 


then  there  are  eigenvalues  .  .  . ,  of  A  such  that 


\ 


-  h?  < 


8  =  1 


1  \\R\\l 
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